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Abstract 

Continuous-time Hopfield network has been an important focus of research area since 1980s whose 
applications vary from image restoration to combinatorial optimization from control engineering to 
associative memory systems. On the other hand, in wireless communications systems literature, power 
control has been intensively studied as an essential mechanism for increasing the system performance. 
A fully distributed power control algorithm (DPCA), called Sigmoid DPCA, is presented by Uykan in 
[10] and [11], which is obtained by discretizing the continuous-time system. In this paper, we present a 
Sigmoid-based " Signal- to-Interference Ratio, (SIR)" balancing dynamic networks, called Sgm"SIR"NN, 
which includes both the Sigmoid power control algorithm (SgmDPCA) and the Hopfield neural networks, 
two different areas whose scope of interest, motivations and settings are completely different. It's shown 
that the Sgm"SIR"NN exhibits features which are generally attributed to Hopfield Networks. Computer 
simulations show the effectiveness of the proposed network as compared to traditional Hopfield Network. 
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I. Introduction 

In neural networks field, continuous-time Hopfield Neural Networks [1] has been an important 
focus of research area since early 1980s whose applications vary from combinatorial optimization 
(e.g. [2], [3] among many others) including traveling salesman problem (e.g. [4], [23] among 
others) to image restoration (e.g. [5]), from various control engineering optimization problems 
including in robotics (e.g. [7] among others) to associative memory systems (e.g. [6], [15] among 
others), etc. For a tutorial and further references about Hopfield NN, see e.g. [13], [8], [20]. 

On the other hand, in the cellular radio systems literature, power control has been intensively 
studied as an essential mechanism for high-capacity cellular networks. In this paper, we construct 
a bridge between two different areas, Hopfield-like Neural Networks and fully distributed power 
control algorithms, whose scope of interest, motivations and settings are completely different. 

Transmitter power control is essential for high-capacity cellular radio systems [24], [28], etc. 
Power Control (PC) problem has drawn much attention since Zander's works on centralized 
[24] and distributed [25] CIR balancing. Carrier-to-Interference-l-noise Ratio (CIR) balancing 
was further investigated by Grandhi et al. [16], [17]. In [19], Foschini and Miljanic considered a 
more general and realistic model, in which a positive receiver noise and a respective target SIR 
were taken into account. The Foschini and Miljanic 's distributed algorithm (FMA) was shown 
to converge either synchronously [19] or asynchronously [27] to a fixed point of a feasible 
system. Based on the FMA, Grandhi et al. [18] suggested distributed constrained power control 
(DCPC) algorithm, in which a transmission upper limit was considered. Some papers focus on 
convergence speed of the PC operation e.g. [26], [21], [9], [22], [11]. It would be very difficult 
to prepare a complete list of all the works on the power control due to the huge amount of 
papers published in the area. 

Starting from the differential equation form of the Sigmoid DPCA in [10] and [11] and relaxing 
the constraints on the positiveness and strict assumptions on the spectral radius of the link gain 
matrix, we establish a bridge from SgmDPCA to the Hopfield-like Neural Networks (NNs). The 
proposed approach yields a Sigmoid basis SIR-balancing NN which exhibits similar features as 
Hopfield NN does. 

Our investigations show that 1) The proposed Sgm"SIR"NN includes both SgmDPCA al- 
gorithm and Hopfield NN as special cases. 2) The Sgm"SIR"NN exhibits features which are 
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generally attributed to Hofield-like recurrent NN. 3) Estalishing an analogy to the SgmDPCA, 
the proposed network as well as the Hopfield Network keeps the fictitious SIR at a target level. 

The paper is organized as follows: The Hopfield Network and PC problem are investigated 
in the same framework in n. Section IE presents the proposed network. Simulation results are 
presented in Section IV followed by Concluding Remarks in Section V. 

II. From Continuous-Time Sigmoid-Basis Power Control Algorithm to 

HOPFIELD-LIKE NNS 

We start with the standard definition of Signal-to-Interferende+Noise-Ratio (SIR) in a cellular 
radio system, in which N mobiles share the same channel (e.g. [24], [25]). 

% - ^ „f , ^ = l,...,iV (1) 

where pi is the transmission power of mobile i, gij is the link gain from mobile j to base i 
involving path loss, shadowing, multi-path fading (as well as the spreading/processing gain in 
case of CDMA transmission [28], etc), and ui is the receiver noise at base station i. 

Without loss of generality, eq.(l) considers the uplink case (from mobile to base) and assume 
that mobile i is assigned to base i at that instant. So, the aim of power control is to determine 
the transmit power for every mobile which keep its SIR (i.e., 7^) at a target value 7*^*. 

Defining H^j — [H]jj = ^l^aij/aa and Ha — and 77^ = Ij^^i^i/du, one may obtain the eq. 
(1) in matrix form as follows at the solution, i.e. when 7^ = 7*^*. 

(I - H)p = (2) 

where p is the power vector, H is the normalized link gain matrix and 77 is the noise vector. 

Considering the link gain matix H > (i.e., all entries are strictly positive), if the matrix 
(I — H) is nonsingular and all its eigenvalues are strictly positive, then the positivity condition 
(i.e., the solution p* > 0) is met because then (I-H)"^ = I + H + H^ + . . . + H" + . . .. Zander's 
work and all other works in the same line assumes that the spectral radius of matrix H is smaller 
than 1. This is a sufficient condition for a unique positive solution. (For proof, see e.g. Theorem 
3.7 in [30]). The spectral radius of matrix H (N x N) is defined as maxj{|Ai|}^i where Aj are 
eigenvalues of H. In many works which is in the same line as Zander's, it's assumed that there 
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is a network admission algorithm which assures that the spectral radius of the normalized link 
gain matrix H is less than 1. 

In the power control design, the link gain matrix and transmit power are all positive. In what 
follows, we relax the positivity and radial spectral raduis conditions with the intend of having 
multiple equilibrium points to store a set of prototype vectors and then proceed to Hopfield-like 
Neural Networks. 

Multiplying the eq. (1) with from both sides gives 



QiiPi 



N (3) 



Let's define the following fictitious "Signal to Interference Ratio (SIR)" (9i) by rewriting the 
eq.(3) with neural network terminology: 

^ = ^ i = l,...,7V (4) 

where 9i is the defined fictitious "SIR", Xi is the state of the i'th neuron, an is the feedback 
coefficient from its state to its input layer, Wij is the weight from the output of the j'th neuron 
to the input of the j'th neuron. 

From eq.(4), we define the following error signal 

N 

e-i = -auXi + li, where li = k + ^ WijXj, i = 1,. . .,N (5) 

The norm of the error signal vector e = [ei . . . bat]^ can also be used as a performance index. 
In the second part of this work in [12], 11 norm of e is examined in discrete time, and is shown 
to converge to e = under some reasonable conditions both synchronously and asynchronously. 

Prototype vectors are defined as those x's which make 9i = O'^^ = 1, i = l,...,A'^in eq.(4). 
So, from eq.(4) and (5), the prototype vectors make the error signal zero, i.e., = 0, i — 
1,. . . ,N given that Xi^Q and U ^ 0. 

The Fig. l.a shows the network corresponding to the differential equation form of the Sigmoid 
DPCA in [10] and [11]. The design problem in [10] and [11] is to device a stable dynamic 
Multiple Input Multiple Output (MIMO) network with the unique solution of p* = (I — H)?; to 
be achieved as fast as possible. Note that the unique solution balances the SIRs of every mobile 
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in its setting. Our paper extends the analysis to the multiple equilibrium points cases as follows: 
In this paper, we examine 

a) the case where the states (transmit powers) are not bounded, (which is not a practical 
assumption in the power control algorithm). This case corresponds to Fig.l.a and is examined 
in the rest of this section. 

b) the cases where the states (transmit powers) are upper and lower bounded (inspired by the 
fact that the transmit power in the power control is lower and upper bounded in practise). These 
cases are examined in Section III, which correspond to the networks in Fig.l.c 

In all these cases, we examine if the corresponding MIMO systems exhibit similar features 
as traditional continuous Hopfield Network does. 

Writing the differential equation of the network in Fig.l.a in matrix form gives 

i = fi ( - Ax + Wx + b) (6) 

where x shows the derivative of x with respect to time, i.e., x = ^, and /i(-) represents the 
sigmoid function and 



A = 



an 
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In eq.(7), A shows the self-state-feedback matrix, W with zero diagonal shows the inter- 
neurons connection weight matrix, and b is a threshold vector. In (6), the sigmoid function is 
/i(e) = 1 — ij^expi-aie) ' whcrc (Ti > is called the slope of /i(-), which is equal to its derivative 
with respect to its argument at origin 0. 

It's well known that desigining the weight matrix W as a symmetric one yields that all 
eigenvalues are real, which we assume throughout the paper due to the simplicity and brevity 
of its analysis. 

Proposition 1: 

The network in eq. (6) 
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a) has unique equilibrium point and is globally stable if all the eigenvalues of matrix (— A + W) 
are strictly negative, i.e. . Aj < 0, ,i — 1, N where Aj are the eigenvalues of (—A + W). 

b) is unstable in the sense that at least one Xi{t) goes to infinity, if at least one of the 
eigenvalues of matrix (—A + W) is strictly positive, i.e. there exists a A^ > 0. 

c) has infinite number of equlibrium points in the unbounded input space and finite number of 
equilibrium points in a limited specified input space like a hybercube, if the matrix (—A + W) is 
a negative semidefinite matrix (i.e., there exists only zero and negative eigenvalues of (—A + W) 
). So, by properly designing matrices — A, W for a specified input space like a hybercube, a set 
of prototype vectors could be stored on the equilibrium points. 

Proof: 

a) In what follows, we present a Lyapunov function for providing a sufficient condition for 
the stability of the network in (6). 

A Lyapunov function candidate for (6) is: 

l^(x) = -^x^(-yl + W^)x-b^x (8) 

where the matrix (—A + W) is a negative definite matrix which assures that the Lyapunov 
function in (8) is lower bounded. 

The derivative of the Lyapunov function with respect to time gives 

^^^^ ^ ^ ^ + ^^"^ + ^^^^ 

From eq.(6) and (9) 

V{t) = -fr'(i^)i (10) 

where shows the inverse of the sigmoid function /i(-). Since sigmoid function is an 

odd function and is zero if and only if its argument is zero, we obtain from (10) 

< if and only if ||x|| ^ 0, 



(11) 

if and only if | |x| | = 



It's well known that all the eigenvalues of a negative definite matrix is strictly negative and 
is nonsingular (see e.g. [30]). This implies that there is a unique solution for eq.(6), i.e., x^* = 
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(—A + W)~^b. This observation together with the Lyapunov function analysis above (eq. (8) 
and (11), proves part a. 

b) The stability of the system eq.(6) depends only on the matrix (— A + W), and not on the 
vector b: Let the system be stable and let x* be the equilibrium point of eq.(6), i.e., — A + 
W^x* + b = 0. Defining x(t) = x(i) — x*, and replacing it in eq.(6) yields 

i = fi((-A + W)x) (12) 

As the state vector x(i) approaches to the x* in eq.(6), the x approaches to the origin in 
eq.(12). On the other hand, if the network in eq.(6) is unstable, then the network (12) is also 
unstable, which shows that the stability of the system (6) does not depend on the vector b. In 
brief, from stability point of view, it would be enough to examine only the matrix (—A + W), 
and not (b) in eq.(6). 

If the matrix (— A + W) has a positive eigenvalue, then taking the initial state as the corre- 
sponding eigenvector, shown as x^*^', i.e., x{t = 0) = x^*^ gives 

i = fi(Ajx), x(0)=x^'^ (13) 

where shows the positive eigenvalue and /i(-) is the sigmoid function. 
Since /(•) is an odd function, eq.(13) is unstable with > simply because then the sign 
of the d'x/dt is the same as the sign of x(t) in eq.(13), which proves part b. 

c) Let's assume that the matrix (—A + W) is a negative semidefinite matrix, and let's choose 
the same Lyapunov function candidate as in eq. (8), i.e., V{x.) = —^x^{—A + W)x. — b^x. 
Then, the Lyapunov function is lower bounded. Following the steps in part a above yields 



V(t) = -f-^(x)x 



(14) 



which indicates that 



V< 



(15) 



< if and only if ||x|| =^ 0, 
= if and only if | |x| | = 
The equilibrium points of network in eq.(6), which corresponds to Fig. l.a, satisfies the 
following linear equation: 
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(16) 



On the other hand, writing the eq. (4) in matrix form gives 
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Note that in eq.(17) the matrix (W) and (b) are constant and only matrix (A) varies as any 
of the Xi changes because then all 9j's j=l, N, change according to eq.(4). 

Comparing eq.(17) and eq.(16), we choose 6'-^* = 1 without loss of generality and for the sake 
of brevity. The equilibrium points of the network in (16) and those of the network of eq.(17) 
with = = 1 are equal. 



Clearly, if we think of the whole input space, then there are infinite number of equlibrium 
points due to the singularity of matrix (—A + W) in (16). However, if we think of a certain 
input subspace assuming that the prototype vectors are in the corners of a hybercube as in the 
case of many practical applications, and the initial states are within the hybercube, then there is 
a finite number of equlilibrium points within the hybercube. 

■ 

In the anaysis above, we examined the network with no bound on the states, and show that 
the network is stable in the case of multiple equilibrium points in a bounded hybercube when 
there exists both zero and negative eigenvalues. However, in multiple equilibrium case, the 
simulation results suggest that its performance is quite poor especially when the dimension 
increases. Instead, in what follows, we investigate networks with lower and upper bounded 
states (similar as in the power control case where the maximum transmit power is lower and 
upper bounded in practice). Finally, we end up with a sigmoid based "SIR" -balancing network 
which exhibit similar properties as traditional Hopfield neural networks does. 
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III. "SIR"-BALANCING SIGMOID NEURAL NETWORK WITH BOUNDED STATES 

In the power control formulation in section II, it was assumed that either there is no constraints 
on the maximum transmit power (which is an impractical assumption) or the existing unique 
solution is within the minimum and maximum transmit power constraints. In this section, by 
lower and upper bounding the system states of the proposed networks and relaxing the positivity 
condition, we present a Sigmoid based SIR-balancing networks which exhibits similar features 
as Hopfield NN does. 

In practice, in power control the positive transmit power can not be arbitarily small and large. 
So, writing eq.(l) with the minimum and maximum power constraints gives 

gu Txiax{pmin, niin{p 

max ) 

Pi}} ■ 1 AT /10^ 

where Pmin and Pmax is the minimum and maximum transmit powers. The SIR model in (18) 
can be further written in a more generalized equation as follows 

^. = diiV^i) ^ 

I v^AT / \' ' i,...,iv y^Ly) 

where y{-) represents the modeling of lower and upper bounding the transmit power and of 
any other effects e.g. power amplifier, etc. For example, y{pi) = max{pmim''^^T^{Pmax,Pi}} or 
corresponding piecewise linear function y{pi) = \pi + Pmax\ — \Pi — Pmax\ yields eq.(18). 

By relaxing the positivity conditions in the power control problem in (19) and using sigmoid 
as the bounding function to the states of the proposed network, we define the following fictitious 
"SIR": 

-|f = i^l,...,N (20) 

where /(•) represents the sigmoid function. 

Using sigmoid function in (20) will allow us to design Hopfield-like networks, which includes 
the traditional Hopfield Network as a special case, as will be seen in the following subsection. 

Implementing the above-mentioned upper and lower bounds into the network in Fig.l.a (which 
was originally suggested in [10] and [11] for the power control problem) results in the following 
equation 
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X = fl ( - Af2(x) + Wf2(x) + b) 

Let's define the following error signal for the network eq.(21) 



(21) 



N 

ei = -Ciiif2{xi) + h, where /j = 6^ + ^ Wijf2{xj), i = l,...,N (22) 



Writing (22) in matrix form gives 



e = -Af2(x) + Wf2(x) + b 



(23) 



From eq. (21) and (23), x = fi(e), which shows x = if and only if e = due to the chosen 
sigmoid function So, if = given that 7^ and /j 7^ in eq.(22), then, from eq.(20) 
and (22), 6^ = Of = 1. 

Proposition 2: 

If W is symmetric and (—A + W) is a negative semi-definite matrix, the network in (21) is 
stable, and the error vector e in (23) goes to zero. 
Proof: 

Let's examine the following Lyapunov function candidate for the network eq.(21) 



V(x) = -if2(x^)(-A + W)f2(x) - b^f(a;) 



(24) 



where /2(-) represents the sigmoid function. Note that the Lyapunov function in (24) is lower 
bounded for any x since matrix (— A + W) is negative semi-definite matrix. Next, we examine 
the derivative of the Lyapunov function with respect to time 



Tdi2 



From eq.(21), (-A + W)f2(x) + b = fr^(x). Using that in (25) gives 



(25) 



V{t) = - 



dxi 



Xi 



(26) 
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and 

y^_j2^f-\x,)x, (27) 

Since the inverse function of the sigmoid function is an odd function, fi ^{ ), and is zero if 
and only if its argument is zero, and ^ > in the work regime, we obtain 



< if and only if I Ixl I 0, 

" " (28) 
= if and only if | |x| | = 



From eq.(21) and eq.(23), x — fi(e). Since ) is an increasing odd function and is zero only 
at origin 0, x = if and only if fi(e) = 0. This observation together with eq.(28) completes 
the proof. 

■ 

In what follows, we examine the evolution of an energy function that gives an insight into 
the evolution of the error vector that is defined as the argument of the function fi( ) in a further 

generalized network. As an attempt to better examine the roles of the diagonal matrix A and 
matrix W onto the network dynamic behaviour, let us consider possibly a different function for 
matrix A, denoted as /3(-), in eq.(21) as follows: 

X = fi ( - Af3(x) + Wf2(x) + b) (29) 

where /i(-), /2(-) are sigmoid functions with possibly different (positive) slopes, and /3(-) 
indicates the function implemented to A. 

So, the corresponding error vector is defined as 

e = - Afa (x) + Wf2 (x) + b (30) 
Let's define the energy function for the error vector in (30) as follows 

V{t) = ^fi(e^)fi(e) (31) 
The derivative of the energy function with respect to time gives 
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V = fi(eO 



r cifi(e) 



dt 



= Xi X2 . . . Xn 



t[-aiit + S& 
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,5h 



(32) 



(33) 



(34) 



where matrix J is equal to 
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—dNN 



ShSJ 



(35) 



where ajj > 0. If matrix J in eq.(35) is negative definite, then the error vector in (30) goes to 
zero due to eq.(31) and (32). 
From eq.(35), we observe that 

a) 1^ is seen in every element of raw i. So, it has no effect on the negative definiteness of 
matrix J. 

b) 1^ takes place in all non-diagonal elements, and not in the diagonal elements. From the 
characteristics of the derivative of the sigmoid function, ~ whenever Xi is close to or in 
the saturation regime. This assures that the energy function decreases because y < in (32), 
whenever all the Xi's are in a saturation regime, provided that ^ >> 0. 

c) 1^ is seen on the diagonal elements only. So, taking the observations in a and b into account, 
if /3(-) is chosen a unity function, i.e., /3(xi) = Xj, then the |^ >> for the saturation regime, 
which assures that y < for any x in the saturation regime. 

In the following subsection, we show that choosing /3(-) as a unity function results a network 
which exhibits similar features as traditional Hopfield Network does. 
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A. "SIR" -balancing Sigmoid Neural Network 

From the observations a, b and c above, we choose /3(-) a unity function, i.e., f3{xi) — Xi 
In a smiUlar way as in (20), we define the following "SIR": 



Oi (XijX 



l,...,iV (36) 



where /2(-) is a sigmoid function and all other parameters are defined as in eq.(20). So, the 
only difference is that the self-state-feedback is directly from its state in eq.(36) as contrast to 
its function output in eq.(20). The resulting network is presented in Fig. I.e. 

From Fig.l.c, 

i = fi(- Ax + Wf2(x) + b) (37) 

We will call the network in eq.(37) as Sgm"SIR"NN (Sigmoid "SIR" -balancing neural net- 
work). 

Let's define the following error signal for the network eq.(37) 

N 

ei^-auXi + Ii, where li ^ k ^ Wijf2{xj), i^l,...,N (38) 
Writing (38) in matrix form gives 

e= -Ax + Wf2(x) + b (39) 

From eq. (37) and (39), x — fi(e). So, if Cj = given that Xi ^ and /j 0, then, from 
eq.(36) and (38), 6^ = = 1. 

The Sgm"SIR"NN in eq.(37) includes both the sigmoid power control algorithm in [11], 
which is shown in eq.(6), and the traditional Hopfield NN as special cases. (Taking f2(.) as 
unity function results in sigmoid power control algorithm and taking fi(.) as unity function 
results in Hopfield Network). 

Proposition 3: 

The Sgm"SIR"NN in eq.(37) converges to e = if 
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N 

> M (40) 

Proof: 

Let's choose the Lyapunov function candidate as in (31) for the network eq.(37) as follows 

V{t) = ^fi(e^)fi(e) (41) 

where e is defined in eq.(38). Following the steps from (32) to (35), and taking the observations 
a to c into account, it's seen that if 

N 

Wjjl > I E M (42) 

then 

<0 if and only if ||fi(e)|| 0, 



which completes the proof, since x = fi (e) from eq. (37) and (39) 



(43) 

= if and only if ||fi(e)|| = 



The analysis above is for a diagonally dominant matrix case. In what follows, we examine 
for any positive A and symmetric W case. 

Proposition 4: 

For the Sgm"SIR"NN in eq.(37) with symmetric W, and positive diagonal matrix A, the 
defined error vector in (38) as well as the state vector x(i) stay within a bounded space for any 
time t. 

Proof: 

Let's choose the same energy function in eq.(41). Following the steps from eq.(32) to (35), 
and taking the observations a, b and c into account, we see that: The ^ for any Xi which 
is close to or in the saturation regime due to the characteristics of the derivative of the sigmoid 
function. This assures that the energy function decreases in any saturation regime because < 
in (32) for any x(i) in saturation. Therefore, the norm of (39) do not go to infinity and stay 
within a bounded space. This implies that the x(i) also stays within a bounded space from (39). 
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We show in Proposition 4 that the x(t) stay within a bounded space for any time t. Since 
the number of all possible in-saturation state combinations is finite, which is equal to 2^, one 
may expect that either the network might show a sort of an oscilatory behaviour which would 
never converge to e = or the network will eventually converge to an equilibrium point within 
a finite time, say Td, satisfying e{t > Td) = 0. In what follows we show that the latter one is 
correct, i.e., the network converges to e = 0. 

Proposition 5: 

The Sgm"SIR"NN in eq.(37) with symmetric matrix W and positive diagonal matrix A 
minimizes the following energy function (Lyapunov function) in (44) and exhibits similar features 
as traditional Hopfield Network does. The error vector e in (39) goes to zero. 



where yi — f2{xi) and f2^{yi) represents the inverse of the sigmoid function. ^ 
Proof: 

Taking the derivative of the energy function (Lyapunov function candidate) in (44) with respect 
to time gives 




(44) 



V{t) 




(46) 




(47) 



Since f^Ut r\u)du} = f-^y^), we obtain 



It's shown by Hopfield et al. in [1] that Hopfield network minimizes the energy (Lyapunov) function in eq.(44) 



and its derivative with respect to time is obtained in [1] as follows 



_ _ \ - df ,dxi 2 
dt ~ ^ dxi ^ dt ' 



(45) 
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Vit) = -f2(x)W^ + Ea,,/-^(y,)-b^^ (48) 
= [-f(x)'^W^ + x^A^-b^]^ (49) 

(JjL 



In (48), f2^{yi) = Xi is used. From eq.(37) and (49) 



V{t) = -[fi(x)]^§ (50) 



and finally 



V--T.§^.fl\i^)i^ (51) 

-I H/tAj 'I 

j = l -i 

Since sigmoid /2(.) is an increasing odd function, and ^ > in the work regime. 



V { 



(52) 



<0 if and only if HxHt^^O, 
= if and only if | |x| | = 
Eq. (52) shows that the Lyapunov function decreases at all points other than the equilibrium 
points and does not change only at the equilibrium points where the error vector e in (39) is 
zero vector, because x = fi(e) where /i(-) is sigmoid function. This completes the proof. 

■ 

Corollary 1: 

Note that in the Lyapunov function analysis above, there is no assumption on the eigenvalues 
of the matrix W, and the diagonal matrix A. The only assumption is that W is symmetric and 
A is positive. 

So, from the Lyapunov analysis above for symmetric W and positive A, we conclude that 

1) The Sgm"SIR"NN does not show oscilatory behaviour. (This is because the energy function 
of the Sgm"SIR"NN-vl decreases at all points other than the equilibrium points and does 
not change only at the equilibrium points). 

2) The states for any initial condition converge to one of the equilibrium points depending on 
the initial contition. If in the converged eqiulibrium point, 6i = Of^ = 1, then it corresponds 
to a prototype vector. 
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3) All equilibrium points are potential attractors. Equavalantly, all the attractors are stationary 
points of V{x.). 

Note that the features above are also attributed to the traditional Hopfield NN in [1]. Taking 
fi(.) as unity function results in Hopfield NN. 

From (37), the equilibrium points of the Sgm"SIR"NN are the same as those of corresponding 
Hopfield NN. However, the basins of the attractors of the proposed Sgm"SIR"NN are, in general, 
different than those of the Hopfield NN, as will be seen from the simulation results in section 
IV. 

There are various ways for determining the weight matrix of the Hopfield Networks: Gradient- 
descent supervised learning (e.g. [8]), solving linear inequalities (e.g. [31], [32] among others), 
Hebb learning rule [29], [14] etc. How to design CINR-SgmNN is out of the scope of this paper. 
The methods used for traditional Hopfied NN can also be used for the Sgm"CIR"NN. 

As far as the simulation results in section IV are concerned, for the sake of simplicity and 
brevity, we assume that the desired prototype vectors are orthogonal and we use the following 
design procedure for matrices A, W and b, which is based on Hebb learning ([29]): 

Outer products based network design: 

Let's assume that L desired orthogonal prototype vectors, {dslfL^, are chosen form (—1, +1)^. 
Step 1: Calculate the sum of outer products of the prototype vectors (Hebb Rule, [29]) 



L 



Q = ^d,df (53) 



s=l 



Step 2: Determine the diagonal matrix A and W as follows: 



qii + p if i = j, . . 

= i,j^l,...,N (54) 



if ly^J 

where p is a real number and 



Wij 



i,j^l,...,N (55) 



if i=j, 
Qij if i^j 

where qij shows the entries of matrix Q, N is the dimension of the vector x and L is the 
number of the prototype vectors (N > L > 0). In eq.(54), qu — L from (53) since {d^} is from 
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(— 1, +1)^. It's observed that p = gives relatively good performance, however, by examining 
the nonlinear state equations in eq.(37), it can be seen that the proposed network Sgm"SIR"NN 
contains the prototype vectors at their equilibrium points for a relatively large interval of p thanks 
to the bounding effect of the sigmoid function. 

Another choice of p in (54) is p — N — 2L which yields an — N — L. In what follows we 
show that this choice also assures that {dj}^^^ are the equilibrium points of the networks. 

From (53)-(55) 

L 

[- A + W] = - ( AT - L)I + ^ d,dj - LI (56) 

s=l 

where I represents the identity matrix. 

Since ds e (—1,+!)^, then ||ds||2 = N. Using (56) and the orthogonality properties of the 
set {dslfli gives 

[-A + W]d, = -{N - L)ds + {N- L)ds = (57) 
So, the prototype vectors {dj}j^^ correspond to equilibrium points. 

IV. Simulation Results 

In this section, we present two examples, one with 8 neurons and one with 16 neurons. The 
weight matrices are designed by the outer products-based design above. Traditional Hopfield 
network is used a reference network. The continuous Hopfield Network [1] is 

X = - Af2 (x) + Wf2 (x) + b (58) 

where A, W, b and /2(-) is defined as in eq.(37). 
Example 1: 

In this example, there are 8 neurons. The desired prototype vectors are 



D 



11 1 1-1-1-1-1 
11-1-11 1-1-1 
1-11-11-11-1 



(59) 
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The weight matrices A and W, and the threshold vector b are obtained as follows by using 
the outer-products-based design presented in section III-A and the slopes of sigmoid functions 
/i(-) and /2(-) are set to (7i = 10 and 02 — '2 respectively, and p is chosen as -1. 



A = 21, W = 
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where I shows the identity matrix of dimension N by N. 

The Figure 2 shows the percentages of correctly recovered desired patterns for all possible ini- 
tial conditions x(t = 0) e (—1, +1)^, in the proposed Sgm"SIR"NN as compared to traditional 
Hopfield network. 

Let rrid show the number of prototype vectors and C{N, K), (such that N > K >G), represent 
the combination N, K, which is equal to C{N, K) — jf^^jj^, where ! shows factorial. In our 
simulation, the prototype vectors are from (—1,1)^ as seen above. For initial conditions, we 
alter the sign of K states where K=Q, 1, 2, 3 and 4, which means the initial condition is within 
iT-Hamming distance from the corresponding prototype vector. So, the total number of different 
possible combinations for the initial conditions for this example is 24, 84 and 168 for 1, 2 and 
3 -Hamming distance cases respectively, which could be calculated by x C(8, X), where 
= 3 and K — \,1 and 3. 

As seen from Figure 2, the performance of the proposed network Sgm"SIR"NN is the same as 
that of the continuous Hopfield Network for 1 -Hamming distance case (%100 for both networks) 
and is slightly and noticeably higher than that of the Hopfield Network for 2 and 3 -Hamming 
distance cases respectively. However, it's known that the performance of Hopfield network 
may highly depend on the weight matrices. For example, it's observed that for A = —31, 
the performance of Hopfield Network is slightly better than the proposed network for the 
same weights W and A. So, our test simulation results suggest that the proposed network 
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Sgm"SIR"NN and the Hopfield network, in general, gives comparable performances in many 
cases. To investigate when either one ourperforms the other one would be an interesting future 
research item. 

Example 2: 

The desired prototype vectors are 



D 



(61) 



11 1 1 1 1 1 1-1-1-1-1-1-1-1-1 
11 1 1-1-1-1-11 1 1 1-1-1-1-1 
11-1-11 1-1-11 1-1-11 1-1-1 
1-11-11-11-11-11-11-11-1 
The weight matrices A and W and threshold vector b is obtained as follows by using the 
outer products based design as explained above. For matrix A, p is chosen as -2. The other 
network paramaters are chosen as in example I: ai — 10, (J2 — 2. 



A = 21, 
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w 
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b = 



(62) 



The Figure 3 shows the percentages of correctly recovered desired patterns for all possible 
initial conditions x(t = 0) G (—1,+!)^^, in the proposed Sgm"SIR"NN as compared to 
traditional Hopfield network. 

The total number of different possible combinations for the initial conditions for this example 
is 64, 480 and 2240 and 7280 for 1, 2, 3 and 4-Hamming distance cases respectively, which 
could be calculated by rrid x C(16, K), where rrid — 4 and K — 1, 2, 3 and 4. 

As seen from Figure 3 the performance of the proposed network D-Sgm"SIR"NN is the same 
as that of Hopfield Network for 1, 2 and 3-Hamming distance cases (%100 for both networks). 
The Hopfield network network gave slightly better performance than the proposed network for 
4-Hamming distance case. 

V. Concluding Remarks 

In this paper, starting from the power control algorithm in [11], we present a Sigmoid- 
based "Signal-to-Interference Ratio, (SIR)" balancing dynamic network, called Sgm"SIR"NN, 
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which includes both the Sigmoid power control algorithm (SgmDPCA) and the Hopfield neural 
networks, two different areas whose scope of interest, motivations and settings are completely 
different. The stability of the Sgm"SIR"NN is examined by the proposed Lyapunov functions. 

Starting from the differential equation form of the Sigmoid DPCA in [10] and [11] and relaxing 
the strick restrictions and assumptions on the positiveness and spectral radius of the link gain 
matrix, we establish a link from SgmDPCA to the Hopfield-like NNs. The proposed approach 
yields a Sigmoid basis SIR-balancing NN which exhibits similar features as Hopfield NN does. 
Computer simulations show the effectiveness of the proposed network as compared to traditional 
Hopfield Network. 

Our investigations show that 1) The proposed Sgm"SIR"NN includes both SgmDPCA al- 
gorithm and Hopfield NN as special cases. 2) The Sgm"SIR"NN exhibits features which are 
generally attributed to Hofield-like recurrent NN. 3) Estalishing an analogy to the SgmDPCA, 

the proposed network as well as the Hopfield Network keeps the fictitious SIR at a target level. 

As a continuation of this work, we examine the proposed network in discrete time and compare 
it to discrete Hopfield Networks in [12]. 



The derivative of /(■) is /'(a) = gaa(i^e°-a)2 whose maximum is at the point a = 0, i.e., 
I /'(a) I < 0.5a. So we obtain the following inequality 



Appendix 




cr > 0) has the 



/(a) - f{b) = (a - 6)/' (/.a + (1 - - a)) 



with II e [0, 1] 



\f{a)-m\<k\a-b 



(63) 



where A; = 0.5cr is the global Lipschitz constant of the sigmoid function. 
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Fig. 1. (a) SgmDPCA in [10] and [11], (b) Hopfield Neural Network, (c) Proposed network, Sgm"SIR"NN, (Sgm"SIR"NN 
contains SgmDPCA and Hopfield NN as special cases). 
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Percentage of correctly recovered prototype vectors, N=8 




I Hamming dist.=1 
] Hamming dist.=2 
J Hamming dist.=3 



Hopfield Networks 



Sgm"SIR"NN 



Fig. 2. The figure shows percentage of correctly recovered desired patterns for all possible initial conditions in example 1 for 
the proposed Sgm"SIR"NN as compared to traditional Hopfield network with 8 neurons. 
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Percentage of correctly recovered prototype vectors, N=16 




Hopfield Networks Sgm"SIR"NN 



Fig. 3. The figure shows percentage of correctly recovered desired patterns for all possible initial conditions in example 2 for 
the proposed Sgm"SIR"NN as compared to traditional Hopfield network with 16 neurons. 
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